Processing brain data using autoencoder neural networks

ABSTRACT

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing brain data using autoencoder neural networks. One of the methods includes obtaining brain data captured by one or more sensors characterizing brain activity of a patient; processing the brain data to generate modified brain data that characterizes a predicted local effect of a future treatment on the brain of the patient; processing the modified brain data using an autoencoder neural network to generate reconstructed brain data; and determining, using the reconstructed brain data, a predicted global effect of the future treatment on the brain of the patient.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to the U.S. patent application Ser. No.17/066,171, filed on Oct. 8, 2020, and incorporated herein by referencein its entirety.

This application is also related to the U.S. patent application Ser. No.17/066,178, filed on Oct. 8, 2020, and incorporated herein by referencein its entirety.

BACKGROUND

This specification relates to processing data related to the brain of apatient, e.g., functional magnetic resonance imaging (MRI) data and/ortractography data.

Brain functional connectivity data characterizes, for each of one ormore pairs of locations within the brain of a patient, the degree towhich brain activity in the pair of locations is correlated.

One can gather data related to the brain of the patient by obtaining andprocessing images of the brain of the patient, e.g., using magneticresonance imaging (MM), diffusion tensor imaging (DTI), or functional MMimaging (fMRI). Diffusion tensor imaging uses magnetic resonance imagesto measure diffusion of water in a human brain. One can use the measureddiffusion to generate tractography data, which can include images ofneural tracts and corresponding white matter fibers of the subjectbrain.

Data related to the brain of a single patient can be highly complex andhigh-dimensional, and therefore difficult for a clinician to manuallyinspect and parse, e.g., to plan a surgery or diagnose the patient for abrain disease or mental disorder.

SUMMARY

This specification describes systems implemented as computer programs onone or more computers in one or more locations for processing brain dataof a patient using an autoencoder neural network.

In this specification, an autoencoder neural network is a neural networkthat includes at least two subnetworks: an encoder subnetwork and adecoder subnetwork. The encoder subnetwork is configured to process anetwork input of the autoencoder and to generate an embedding of thenetwork input. The decoder subnetwork is configured to process theembedding of the network input and to generate a reconstructed networkinput. Typically, an autoencoder neural network is trained to generate areconstructed network input that is as similar to the network input aspossible.

In some implementations described in this specification, a system canprocess “modified” brain data using an autoencoder neural network togenerate reconstructed brain data. The modified brain data cancharacterize a predicted local effect of a future treatment on the brainof the patient, and the reconstructed brain data can characterize aglobal effect of the future treatment on the brain of the patient.Because a local effect of a treatment (i.e., an effect of the treatmenton a region of the brain that is local to a target location of thetreatment) can be easier to predict than the global effect of thetreatment, the system can leverage the autoencoder neural network togenerate an accurate prediction of the global effects of a treatmentbefore the treatment is provided. The system can thus allow a user,e.g., a clinician or other medical professional, to determine atreatment for the patient that will be, or is more likely to be, safeand effective.

In some other implementations described in this specification, a systemcan process “desired” brain data using an inverted autoencoder neuralnetwork to generate “roadmap” brain data. The desired brain data cancharacterize a desired global effect of a future treatment on the brainof the patient, and the roadmap brain data can characterize a localeffect of the future treatment on the brain of the patient. Because alocal effect of a treatment can be easier to predict than the globaleffect of the treatment, the system can leverage the invertedautoencoder neural network to generate roadmap brain data from which afuture treatment can be determined that, if provided to the patient,will actualize, or be more likely to actualize, the desired globaleffect of the future treatment. That is, the system or a user of thesystem, e.g., a clinician or other medical professional, can use theroadmap brain data to determine parameters of the future treatment(e.g., a location in the brain of the patient to target with thetreatment) such that the future treatment will be, or is more likely tobe, safe and effective.

In some other implementations described in this specification, a systemcan process brain data of a patient using an autoencoder neural networkto generate reconstructed brain data, and then process the reconstructedbrain data to determine whether the original brain data of the patientis anomalous, i.e., outside of a normal range of values. For example,the system can determine that the brain data is anomalous if a measureof the difference between the brain data and the reconstructed braindata exceeds a threshold. That is, the system can compare the originalbrain data to the reconstructed brain data (where the reconstructedbrain data incorporates information from other patients learned duringthe training of the autoencoder neural network) to identify anomalies inthe original brain data. In some such implementations, the system canfurther identify a subset of the original brain data that is anomalous,e.g., a subset corresponding to a particular region of the brain of thepatient. The system can thus allow a user, e.g., a clinician or othermedical professional, to quickly identify anomalies in the brain datathat might indicate that the patient has a brain disease and,optionally, a region of the brain that might be the target for atreatment of the brain disease.

In this specification, brain data can be any data characterizing thebrain of a patient. For example, brain data can include one or both ofi) direct measurement data of the brain of the patient, e.g., images ofthe brain collected using brain imaging techniques, or ii) data that hasbeen derived or generated from initial measurement data of the brain ofthe patient, e.g., correlation matrices.

Particular embodiments of the subject matter described in thisspecification can be implemented so as to realize one or more of thefollowing advantages.

As described above, a set of brain data characterizing the brain of asingle patient can often be incredibly large and complicated, and thusit can be difficult and time consuming for a user to extract usefulinformation from the set of brain data. Using techniques described inthis specification, a system can process brain data of a patient togenerate a clinically-relevant network output that can be used by a userto provide safe and effective care for the patient. For example, thesystem can allow the user to determine a treatment for the patientsignificantly more accurately and significantly more efficiently (e.g.,in less time or using fewer computational, memory, or network resources)than if the user manually reviewed the large corpus of brain data. Morespecifically, a correlation matrix, e.g., a correlation matrix of fMRIdata, of the brain of a patient can be a matrix with more than a hundredelements, more than a thousand elements, or more typically more than ahundred thousand elements. When a set or series of such correlationmatrices are being considered, a system for producing and/or analyzingsuch matrices may be processing more than a million elements. By usingtechniques described in this specification, a system can reduce thenumber of elements being considered by the user to produce clinicallyrelevant data, e.g., a recommendation for a specific action, within 10minutes, within 5 minutes, within 3 minutes, within a minute, within 30seconds or within 5 seconds.

As described above, the long-term, global effects of a treatment on thebrain of the patient can be very difficult to predict, even byexperienced medical professionals. Using techniques described in thisspecification, a system can use training data representing the clinicaloutcomes of treatments provided to a large number of patients, e.g.,more than a hundred, more than a thousand, more than a hundred thousandor more than a million patients, to train an autoencoder neural networkto accurately predict the outcome of a particular treatment on the brainof a particular patient. Similarly, a system can use training data totrain an autoencoder neural network to accurately characterize whattreatment will produce, or is likely to produce, a desired outcome onthe brain of the patient. The autoencoder neural networks describedherein can thus learn complex, nonlinear relationships between differentregions of the brain and different treatments thereof, allowing a userto pursue safe and effective treatments for a patient in a way thatcannot be done by inspecting the brain data of the patient alone.

Using techniques described in this specification, a system can quicklyidentify one or more regions, e.g., parcellations, in the brain of thepatient whose brain data is outside a normal range, and therefore mightbe an indicator of a brain disease. The system can then display datacharacterizing the identified regions to the user, so that the user isnot forced to search through and analyze a large amount of data that isnot clinically relevant. Therefore, the amount of time that a user mustspend to discover the portion of the brain data that is useful to theuser can be drastically reduced, resulting in improved outcomes forpatients, users and/or clinicians, especially when effective carerequires time sensitive investigations.

The details of one or more embodiments of the subject matter of thisspecification are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages of thesubject matter will become apparent from the description, the drawings,and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A and FIG. 1B are block diagrams that illustrate an examplecomputer system for use in processing medical images.

FIG. 2A is a diagram of an example autoencoder neural network system.

FIG. 2B is a diagram of an example inverted autoencoder neural networksystem.

FIG. 3 is a diagram of an example anomaly detection system.

FIG. 4 is a flowchart of an example process for processing brain datausing an autoencoder neural network.

FIG. 5 is a flowchart of an example process for processing brain datausing an inverted autoencoder neural network.

FIG. 6 is a flowchart of an example process for identifying anomalousbrain data using an autoencoder neural network.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

This specification describes a system that can brain data of a patientusing an autoencoder neural network.

FIGS. 1A and 1B are block diagrams of a general-purpose computer system100 upon which one can practice arrangements described in thisspecification. The following description is directed primarily to acomputer server module 101. However, the description applies equally orequivalently to one or more remote terminals 168.

As seen in FIG. 1A, the computer system 100 includes: the servercomputer module 101; input devices such as a keyboard 102, a pointerdevice 103 (e.g., a mouse), a scanner 126, a camera 127, and amicrophone 180; and output devices including a printer 115, a displaydevice 114 and loudspeakers 117. An external Modulator-Demodulator(Modem) transceiver device 116 may be used by the computer server module101 for communicating to and from the remote terminal 168 over acomputer communications network 120 via a connection 121 and aconnection 170. The aforementioned communication can take place betweenthe remote terminal 168 and “the cloud” which in the present descriptioncomprises at least the one server module 101. The remote terminal 168typically has input and output devices (not shown) which are similar tothose described in regard to the server module 101. The communicationsnetwork 120 may be a wide-area network (WAN), such as the Internet, acellular telecommunications network, or a private WAN. Where theconnection 121 is a telephone line, the modem 116 may be a traditional“dial-up” modem. Alternatively, where the connection 121 is a highcapacity (e.g., cable) connection, the modem 116 may be a broadbandmodem. A wireless modem may also be used for wireless connection to thecommunications network 120.

The computer server module 101 typically includes at least one processorunit 105, and a memory unit 106. For example, the memory unit 106 mayhave semiconductor random access memory (RAM) and semiconductor readonly memory (ROM). The remote terminal 168 typically includes as leastone processor 169 and a memory 172. The computer server module 101 alsoincludes a number of input/output (I/O) interfaces including: anaudio-video interface 107 that couples to the video display 114,loudspeakers 117 and microphone 180; an I/O interface 113 that couplesto the keyboard 102, mouse 103, scanner 126, camera 127 and optionally ajoystick or other human interface device (not illustrated); and aninterface 108 for the external modem 116 and printer 115. In someimplementations, the modem 116 may be incorporated within the computermodule 101, for example within the interface 108. The computer module101 also has a local network interface 111, which permits coupling ofthe computer system 100 via a connection 123 to a local-areacommunications network 122, known as a Local Area Network (LAN). Asillustrated in FIG. 1A, the local communications network 122 may alsocouple to the wide network 120 via a connection 124, which wouldtypically include a so-called “firewall” device or device of similarfunctionality. The local network interface 111 may include an Ethernetcircuit card, a Bluetooth® wireless arrangement or an IEEE 802.11wireless arrangement; however, numerous other types of interfaces may bepracticed for the interface 111.

The I/O interfaces 108 and 113 may afford either or both of serial orparallel connectivity; the former may be implemented according to theUniversal Serial Bus (USB) standards and having corresponding USBconnectors (not illustrated). Storage memory devices 109 are providedand typically include a hard disk drive (HDD) 110. Other storage devicessuch as a floppy disk drive and a magnetic tape drive (not illustrated)may also be used. An optical disk drive 112 is typically provided to actas a non-volatile source of data. Portable memory devices, such opticaldisks (e.g., CD-ROM, DVD, Blu-ray Disc™), USB-RAM, portable, externalhard drives, and floppy disks, for example, may be used as appropriatesources of data to the system 100.

The components 105 to 113 of the computer module 101 typicallycommunicate via an interconnected bus 104 and in a manner that resultsin a conventional mode of operation of the computer system 100 known tothose in the relevant art. For example, the processor 105 is coupled tothe system bus 104 using a connection 118. Likewise, the memory 106 andoptical disk drive 112 are coupled to the system bus 104 by connections119.

The techniques described in this specification may be implemented usingthe computer system 100, e.g., may be implemented as one or moresoftware application programs 133 executable within the computer system100. In some implementations, the one or more software applicationprograms 133 execute on the computer server module 101 (the remoteterminal 168 may also perform processing jointly with the computerserver module 101), and a browser 171 executes on the processor 169 inthe remote terminal, thereby enabling a user of the remote terminal 168to access the software application programs 133 executing on the server101 (which is often referred to as “the cloud”) using the browser 171.In particular, the techniques described in this specification may beeffected by instructions 131 (see FIG. 1B) in the software 133 that arecarried out within the computer system 100. The software instructions131 may be formed as one or more code modules, each for performing oneor more particular tasks. The software may also be divided into twoseparate parts, in which a first part and the corresponding code modulesperforms the described techniques and a second part and thecorresponding code modules manage a user interface between the firstpart and the user.

The software may be stored in a computer readable medium, including thestorage devices described below, for example. The software is loadedinto the computer system 100 from the computer readable medium, and thenexecuted by the computer system 100. A computer readable medium havingsuch software or computer program recorded on the computer readablemedium is a computer program product. Software modules for that executetechniques described in this specification may also be distributed usinga Web browser.

The software 133 is typically stored in the HDD 110 or the memory 106(and possibly at least to some extent in the memory 172 of the remoteterminal 168). The software is loaded into the computer system 100 froma computer readable medium, and executed by the computer system 100.Thus, for example, the software 133, which can include one or moreprograms, may be stored on an optically readable disk storage medium(e.g., CD-ROM) 125 that is read by the optical disk drive 112. Acomputer readable medium having such software or computer programrecorded on it is a computer program product.

In some instances, the application programs 133 may be supplied to theuser encoded on one or more CD-ROMs 125 and read via the correspondingdrive 112, or alternatively may be read by the user from the networks120 or 122. Still further, the software can also be loaded into thecomputer system 100 from other computer readable media. Computerreadable storage media refers to any non-transitory tangible storagemedium that provides recorded instructions and/or data to the computersystem 100 for execution and/or processing. Examples of such storagemedia include floppy disks, magnetic tape, CD-ROM, DVD, Blu-ray™ Disc, ahard disk drive, a ROM or integrated circuit, USB memory, amagneto-optical disk, or a computer readable card such as a PCMCIA cardand the like, whether or not such devices are internal or external ofthe computer module 101. Examples of transitory or non-tangible computerreadable transmission media that may also participate in the provisionof software, application programs, instructions and/or data to thecomputer module 101 include radio or infra-red transmission channels aswell as a network connection to another computer or networked device,and the Internet or Intranets including e-mail transmissions andinformation recorded on Websites and the like.

The second part of the application programs 133 and the correspondingcode modules mentioned above may be executed to implement one or moregraphical user interfaces (GUIs) to be rendered or otherwise representedupon the display 114. For example, through manipulation of the keyboard102 and the mouse 103, a user of the computer system 100 and theapplication may manipulate the interface in a functionally adaptablemanner to provide controlling commands and/or input to the applicationsassociated with the GUI(s). Other forms of functionally adaptable userinterfaces may also be implemented, such as an audio interface utilizingspeech prompts output via the loudspeakers 117 and user voice commandsinput via the microphone 180.

FIG. 1B is a detailed schematic block diagram of the processor 105 and a“memory” 134. The memory 134 represents a logical aggregation of all thememory modules (including the HDD 109 and semiconductor memory 106) thatcan be accessed by the computer module 101 in FIG. 1A.

When the computer module 101 is initially powered up, a power-onself-test (POST) program 150 can execute. The POST program 150 can bestored in a ROM 149 of the semiconductor memory 106 of FIG. 1A. Ahardware device such as the ROM 149 storing software is sometimesreferred to as firmware. The POST program 150 examines hardware withinthe computer module 101 to ensure proper functioning and typicallychecks the processor 105, the memory 134 (109, 106), and a basicinput-output systems software (BIOS) module 151, also typically storedin the ROM 149, for correct operation. Once the POST program 150 has runsuccessfully, the BIOS 151 can activate the hard disk drive 110 of FIG.1A. Activation of the hard disk drive 110 causes a bootstrap loaderprogram 152 that is resident on the hard disk drive 110 to execute viathe processor 105. This loads an operating system 153 into the RAMmemory 106, upon which the operating system 153 commences operation. Theoperating system 153 is a system level application, executable by theprocessor 105, to fulfil various high-level functions, includingprocessor management, memory management, device management, storagemanagement, software application interface, and generic user interface.

The operating system 153 manages the memory 134 (109, 106) to ensurethat each process or application running on the computer module 101 hassufficient memory in which to execute without colliding with memoryallocated to another process. Furthermore, the different types of memoryavailable in the system 100 of FIG. 1A must be used properly so thateach process can run effectively. Accordingly, the aggregated memory 134is not intended to illustrate how particular segments of memory areallocated (unless otherwise stated), but rather to provide a generalview of the memory accessible by the computer system 100 and how such isused.

As shown in FIG. 1B, the processor 105 includes a number of functionalmodules including a control unit 139, an arithmetic logic unit (ALU)140, and a local or internal memory 148, sometimes called a cachememory. The cache memory 148 typically includes a number of storageregisters 144-146 in a register section. One or more internal busses 141functionally interconnect these functional modules. The processor 105typically also has one or more interfaces 142 for communicating withexternal devices via the system bus 104, using a connection 118. Thememory 134 is coupled to the bus 104 using a connection 119.

The application program 133 includes a sequence of instructions 131 thatmay include conditional branch and loop instructions. The program 133may also include data 132 which is used in execution of the program 133.The instructions 131 and the data 132 are stored in memory locations128, 129, 130 and 135, 136, 137, respectively. Depending upon therelative size of the instructions 131 and the memory locations 128-130,a particular instruction may be stored in a single memory location asdepicted by the instruction shown in the memory location 130.Alternately, an instruction may be segmented into a number of parts eachof which is stored in a separate memory location, as depicted by theinstruction segments shown in the memory locations 128 and 129.

In general, the processor 105 is given a set of instructions which areexecuted therein. The processor 105 waits for a subsequent input, towhich the processor 105 reacts to by executing another set ofinstructions. Each input may be provided from one or more of a number ofsources, including data generated by one or more of the input devices102, 103, data received from an external source 173, e.g., a brainimaging device 173 such such as an Mill or DTI scanner, across one ofthe networks 120, 122, data retrieved from one of the storage devices106, 109 or data retrieved from a storage medium 125 inserted into thecorresponding reader 112, all depicted in FIG. 1A. The execution of aset of the instructions may in some cases result in output of data.Execution may also involve storing data or variables to the memory 134.

Some techniques described in this specification use input variables 154,e.g., data sets characterizing the brain of a patient, which are storedin the memory 134 in corresponding memory locations 155, 156, 157. Thetechniques can produce output variables 161, which are stored in thememory 134 in corresponding memory locations 162, 163, 164. Intermediatevariables 158 may be stored in memory locations 159, 160, 166 and 167.

Referring to the processor 105 of FIG. 1B, the registers 144, 145, 146,the arithmetic logic unit (ALU) 140, and the control unit 139 worktogether to perform sequences of micro-operations needed to perform“fetch, decode, and execute” cycles for every instruction in theinstruction set making up the program 133. Each fetch, decode, andexecute cycle can include i) a fetch operation, which fetches or readsan instruction 131 from a memory location 128, 129, 130; ii) a decodeoperation in which the control unit 139 determines which instruction hasbeen fetched; and iii) an execute operation in which the control unit139 and/or the ALU 140 execute the instruction.

Thereafter, a further fetch, decode, and execute cycle for the nextinstruction may be executed. Similarly, a store cycle may be performedby which the control unit 139 stores or writes a value to a memorylocation 132.

Each step or sub-process in the techniques described in thisspecification may be associated with one or more segments of the program133 and is performed by the register section 144, 145, 146, the ALU 140,and the control unit 139 in the processor 105 working together toperform the fetch, decode, and execute cycles for every instruction inthe instruction set for the noted segments of the program 133. Althougha cloud-based platform has been described for practicing the techniquesdescribed in this specification, other platform configurations can alsobe used. Furthermore, other hardware/software configurations anddistributions can also be used for practicing the techniques describedin this specification.

FIG. 2A is a diagram of example autoencoder neural network system 200.The autoencoder neural network system 200 is an example of systemsimplemented as computer programs on one or more computers in one or morelocations, in which the systems, components, and techniques describedbelow can be implemented.

The autoencoder neural network system 200 is configured to processpatient brain data 202 and to generate reconstructed patient brain data222 corresponding to the patient brain data 202.

The autoencoder neural network system 200 includes an encoder subnetwork210 and a decoder subnetwork 220. The encoder subnetwork 210 isconfigured to process the brain data 202 and to generate an embedding212 of the brain data 202. The decoder subnetwork 220 is configured toprocess the embedding 212 of the brain data 202 and to generate thereconstructed brain data 222. In this specification, an embedding is anordered collection of numeric values that represents an input in aparticular embedding space; e.g., an embedding can be a vector offloating point or other numeric values that has a fixed dimensionality.In this specification, reconstructed brain data is brain data that hasbeen estimated using an embedding of the brain data, e.g., by processingthe embedding using a decoder subnetwork of an autoencoder neuralnetwork.

Generally, the embedding 212 has a lower dimensionality than the braindata 202, while the reconstructed brain data 222 has the samedimensionality as the patient brain data 202. For example, thedimensionality of the embedding 212 can be 1/10^(th), 1/100^(th), or1/1000^(th) the size of the dimensionality of the brain data 202. Thatis, there is a loss of information when the brain data 202 is processedby the encoder subnetwork 210 to generate the embedding 212, and so thereconstructed brain data 222 is only an approximation of the originalbrain data 202. During training, the encoder subnetwork 202 can learn toencode as much information from the brain data 202 as possible into theembedding 212, and the decoder subnetwork 220 can learn to reconstructthe brain data 202 to generate the reconstructed brain data 222 suchthat the reconstructed brain data 222 reflects the information encodedinto the embedding 212.

That is, the autoencoder neural network system 200 can be configuredthrough training to generate reconstructed patient brain data 222 suchthat a difference between the patient brain data 202 and thereconstructed patient brain data 202 is reduced below a threshold orminimized. For example, a training system can process training examplesthat each include brain data corresponding to respective differentpatients using the autoencoder neural network system 200 to generaterespective sets of reconstructed brain data. For each training example,the training system can determine a reconstruction error thatcharacterizes a difference between i) the brain data of the respectivepatient and ii) the corresponding reconstructed brain data. For example,the reconstruction error can be the L₁ or L₂ distance between the braindata and the reconstructed brain data, or squared versions thereof. Asanother example, the reconstruction error can be the root mean squarederror between the brain data and the reconstructed brain data.

The training system can then backpropagate the reconstruction errorthrough the autoencoder neural network system 200 to determine an updateto the values of the parameters of the autoencoder neural network system200, e.g., using gradient descent. For example, the training system candetermine an update to the values of the parameters of both the encodersubnetwork 210 and the decoder subnetwork 220 (i.e., the encodersubnetwork 210 and the decoder subnetwork 220 can be trainedconcurrently). In other words, the training system can train theautoencoder neural network system 200 in an unsupervised manner, i.e.,using training examples that do not include a ground-truth signal, e.g.,a ground-truth embedding 212 of the brain data of the training example.

The brain data 202 can include any data characterizing the brain of thepatient. For example, the brain data 202 can include one or more ofblood-oxygen-level-dependent imaging data, fMRI data, or EEG datacaptured from the brain of the patient. Instead or in addition, thebrain data 202 can include data that has been generated using respectiveraw data captured from the brain of the patient. For example, the braindata 202 can include correlation data that characterizes, for each pairof regions in the brain of the patient, a degree of correlation betweenthe respective brain activity of the regions in the brain of thepatient. As a particular example, each region can be a parcellation inthe brain of the patient, as defined by a brain atlas.

In this specification, a parcellation is a predefined region of thebrain. For example, a parcellation can be defined by boundaries on athree-dimensional volume of the brain. A parcellation can be definedsuch that the neurons in the parcellation are functionally similaraccording to one or more criteria. For example, a set of parcellationscan be defined according to changes in cortical architecture, function,connectivity, and/or topography.

In this specification, a brain atlas is data that defines one or moreparcellations of a brain of a patient, e.g., by defining in a commonthree-dimensional coordinate system the coordinates of the outline ofthe parcellation or the volume of the parcellation. As another example,the brain data 202 can include tractography data that characterizesneural tracts connecting pairs of regions in the brain of the patient,e.g., pairs of parcellations in the brain of the patient.

In some implementations, the autoencoder neural network system 200 cansimulate the effect of treatment on the brain of the patient.

For example, the autoencoder neural network system 200 can obtainmodified brain data 230 that characterizes a predicted local effect of afuture treatment on the brain of the patient. That is, the modifiedbrain data 230 has been generated by modifying real brain data capturedfrom the brain of the patient in order to reflect the predicted localeffect of the future treatment. In this specification, a local effect ofa treatment is an effect that is local to a target location of thetreatment. For example, the local effect of a treatment can be theeffect of the treatment on the parcellation in the brain of the patientat which the treatment is targeted. As another example, the local effectof a treatment can be the effect of the treatment on the region of thebrain within a threshold distance of the location at which the treatmentis targeted, e.g., within a millimeter or a centimeter of the targetlocation.

A system (e.g., the autoencoder neural network system 200 or an externalsystem) can generate the modified brain data 230 by identifying a targetlocation in the brain of the patient, and modifying the brain data ofthe patient corresponding to the target location, and optionally one ormore neighboring locations to the target location. The system can modifythe brain data corresponding to the target location according to how thefuture treatment is expected to modify the brain data, according to theparameters of the future treatment. As a particular example, the systemcan modify the values in a set of correlation data (e.g., as representedby a correlation matrix in FIG. 2A) corresponding to the particularparcellation that is the target of the treatment, e.g., by eitherincreasing or decreasing the identified correlation between the brainactivity in the particular parcellation and the brain activity in one ormore other parcellations in the brain of the patient.

The future treatment can be any appropriate treatment on the brain ofthe patient. For example, the future treatment can be a drug therapythat may be provided to the patient and that targets a particularlocation in the brain of the patient. As another example, the futuretreatment can be a surgery on the target location in the brain of thepatient. As another example, the future treatment can be transcranialmagnetic stimulation (TMS) targeted at the target location in the brainof the patient.

The autoencoder neural network system 200 can process the modified braindata 230 to generate final brain data 240. In particular, theautoencoder neural network system 200 can i) process the modified braindata 230 using the encoder subnetwork 210 to generate an embedding ofthe modified brain data 230, and ii) process the embedding of themodified brain data 230 using the decoder subnetwork 220 to generate thefinal brain data 240.

The final brain data 240 characterizes a predicted global effect of thefuture treatment on the brain of the patient. In this specification, aglobal effect of a treatment is an effect of the treatment on one ormore locations in the brain of the patient that are not local to atarget location of the treatment. For example, the global effect of atreatment can be the effect of the treatment on the entire brain of thepatient. As another example, the global effect of a treatment can be theeffect of the treatment on one or more neighboring locations of thetarget location in the brain of the patient, e.g., one or moreneighboring parcellations of the target parcellation.

In other words, a local effect of a treatment can be any effect by whichan intervention on a specific parcellation in the brain of the patientdirectly affects the functioning of the specific parcellation. A globaleffect of the treatment can characterize how the treatment changes thefunctioning of one or more networks of parcellations in the brain of thepatient.

As a particular example, if the modified brain data 230 includesmodified correlation data as described above, then the final brain data240 can include correlation data that reflects, for one or moreparticular parcellations that were not the target parcellation of thetreatment (e.g., for every parcellation in the brain of the patient),how the treatment will affect the correlation between the brain activityin the particular parcellation and the brain activity in one or moreother parcellations in the brain of the patient.

The output of the autoencoder neural network system 200 can representthe global effect of the future treatment because of how the autoencoderneural network system 200 has been trained. During training, theautoencoder neural network system 200 receives brain data 202corresponding to untreated patients, i.e., patients who have notrecently undergone brain treatments and whose brain data 202 thereforedoes not reflect the local effects of a treatment. The encodersubnetwork 210 learns to encode the most useful information in the braindata 202 into the brain data embeddings 212. The decoder subnetwork 210learns to generate, using the brain data embeddings 212, reconstructedbrain data 222 that imitates the structure of the brain data 202 ofthese untreated patients. Therefore, when provided modified brain data230 that does reflect a local effect of a treatment, e.g., the effectimmediately after the treatment is provided, the encoder subnetwork 210can generate an embedding that encodes the information from the localeffect in the target location of the treatment. The decoder subnetwork220 can then generate reconstructed brain data 240 that imitates braindata that i) includes the local effect in the target location of thetreatment and ii) corresponds to an untreated patient. That is, thefinal brain data 240 reflects how the rest of the brain data outside ofthe target location would be structured for an untreated patient, giventhat the local effect is present in the target location, i.e., how thelocal effect might propagate to the rest of the brain data.

The final brain data 240 can be used to determine what the global effectof the future treatment will be if the treatment is provided to thebrain of the patient, e.g., whether the future treatment will be safeand/or effective. For example, a user of the autoencoder neural networksystem 200 can analyze the final brain data 240 to determine whether thefuture treatment should be provided to the patient. As another example,the final brain data 240 can be processed by an anomaly detectionengine, e.g., the anomaly detection engine 330 depicted in FIG. 3, todetermine whether the final brain data 240 is anomalous. That is, theanomaly detection engine can determine whether providing the treatmentwill cause one or more anomalies in the brain of the patient. Thisprocess is described in more detail below with reference to FIG. 3.

In some cases, the local effect of the future treatment on the brain ofthe patient might reflect a short-term effect of the future treatment,e.g., the effect of the treatment within seconds, minutes, hours, ordays after the treatment has been provided to the brain of the patient.The global effect of the future treatment on the brain of the patientmight reflect the long-term effect of the future treatment, e.g., theeffect of the treatment days, weeks, months, or years after thetreatment has been provided to the brain of the patient. Thus, a usercan use the autoencoder neural network 200 to predict the long-termeffect of the future treatment, in an effort to ensure the safety andefficacy of the treatment.

In some implementations, the autoencoder neural network system 200 canprocess multiple different sets of modified brain data 230, eachcorresponding to a respective different future treatment to generaterespective sets of final brain data 240. Each future treatment can havedifferent parameters, e.g., different strengths, different doses,different schedules, or different target locations. An external systemcan then help determine, from the respective sets of final brain data240, which future treatments will be, or will likely be, the most safeand/or effective according to the reconstructed brain data, andtherefore which future treatment a clinician should considerrecommending be provided to the patient.

As a particular example, a system can process multiple different sets ofmodified brain data 230 using the autoencoder neural network system 200in an attempt to generate a particular set of “desired” brain data. Thatis, the system processes the multiple different sets of modified braindata 230, generating a respective set of final brain data 240 for eachset of modified brain data 230, in order to identify a particular set ofmodified brain data 230 whose corresponding set of final brain data 240matches, or is closest to matching, the set of desired brain data. Thus,the particular set of modified brain data 230 represents the localeffect of a treatment whose global effect is the set of desired braindata. For example, the system can use a “brute force” approach toidentifying the particular set of modified brain data 230; that is, thesystem can systematically process many different sets of modified braindata 230 (e.g., using a grid search) until identifying the particularset of modified brain data 230.

As another example, the system can determine an initial set of modifiedbrain data 230 and process the initial set of modified brain data usingthe autoencoder neural network system 200. The system can determine adifference between i) the final brain data 240 generated using theinitial set of modified brain data 230 and ii) the desired brain data.The system can then backpropagate the difference through the autoencoderneural network system 200 in order to generate an update to the initialmodified brain data 230 that will decrease the error. That is, insteadof updating the parameters of the neural network system 200 during thebackpropagation, the system updates the input to the autoencoder neuralnetwork system 200, generating a new set of modified brain data 230. Thesystem can then process the new set of modified brain data 230 using theautoencoder neural network system 200 and determine another update tothe input that will further decrease the difference, i.e., that willcause the corresponding final brain data 240 to be even closer to thedesired brain data. The system can iteratively repeat this process untilidentifying the particular set of modified brain data that generates thedesired brain data. For example, the system can perform a predeterminednumber of iterations, or repeat the process until a difference betweenthe generated final brain data 240 and the desired brain data fallsbelow a predetermined threshold.

In some implementations, the system can perform the above iterativeprocess multiple times with different “seed” initial sets of modifiedbrain data 230, and, after completing each iterative process, select theparticular modified set of brain data that generates final brain data240 that is closest to the desired brain data.

Other example techniques for determining a set of modified brain datathat will, or is likely to, effectuate a particular desired result inthe brain of the patient are discussed below with reference to FIG. 2B.

FIG. 2B is a diagram of example inverted autoencoder neural networksystem 250. The inverted autoencoder neural network system 250 is anexample of systems implemented as computer programs on one or morecomputers in one or more locations, in which the systems, components,and techniques described below can be implemented.

The inverted autoencoder neural network system 250 is an invertedversion of the autoencoder neural network 200 depicted in FIG. 2A. Inthis specification, an inverted neural network is a neural network thatinverts one or more operations of a subject neural network such that theinverted neural network maps the outputs of the subject neural networkto the inputs of the subject neural network. Similarly to theautoencoder neural network system 200 depicted in FIG. 2A, the invertedautoencoder neural network system 250 is configured to process braindata to generate reconstructed brain data.

The autoencoder neural network system includes an inverted decodersubnetwork 260 and an inverted encoder subnetwork 270. The inverteddecoder subnetwork 260 is configured to process the brain data togenerate an embedding of the brain data, and the inverted encodersubnetwork 270 is configured to process the embedding of the brain datato generate reconstructed brain data. The inverted decoder subnetwork260 can be an inverted version of the decoder subnetwork 220 depicted inFIG. 2A. Similarly, the inverted encoder subnetwork 270 can be aninverted version of the encoder subnetwork 210 depicted in FIG. 2A.

The inverted autoencoder neural network system 250 can be determinedusing the autoencoder neural network system 200, after the autoencoderneural network system 200 has been trained.

The architecture of the inverted autoencoder neural network system 250can be the inverse of the architecture of the autoencoder neural networksystem 200. If the autoencoder neural network system 200 has N neuralnetwork layers, then the inverted autoencoder neural network system 250can also have N neural network layers, where the i^(th) neural networklayer of the inverted autoencoder neural network system 250 correspondsto the (N−i+1)^(th) neural network layer of the autoencoder neuralnetwork system 200. The architecture of the i^(th) neural network layerof the inverted autoencoder neural network system 250 can be an invertedversion of the architecture of the (N−i+1)^(th) neural network layer ofthe autoencoder neural network system 200. That is, if the (N−i+1)^(th)neural network layer of the autoencoder neural network system 200receives a layer input of a first dimensionality (e.g.,L_(i)×W_(i)×H_(i)) and generates a layer output of a seconddimensionality (e.g., L_(o)×W_(o)×H_(o)), then the i^(th) neural networklayer of the inverted autoencoder neural network system 250 receives alayer input of the second dimensionality and generates a layer output ofthe first dimensionality.

The parameter values of the inverted autoencoder neural network system250 can be determined using the trained parameter values of theautoencoder neural network system 200, by any appropriate process.

In some implementations, the inverted autoencoder neural network system250 can be used to identify a treatment that, if provided to a patient,would effect a particular desired outcome on the brain of the patient.

For example, the inverted autoencoder neural network system 250 canobtain desired brain data 252 that characterizes a desired global effectof a future treatment on the brain of a patient. That is, the desiredbrain data 252 represents a desired state of the brain of the patientafter the treatment has been provided to the patient.

In some implementations, a system (e.g., the inverted autoencoder neuralnetwork system 250 or an external system) can determine the desiredbrain data 252 of the patient using brain data corresponding to adifferent second patient that reflect the desired global effect of thefuture treatment. For example, the desired brain data 252 can be thesame as the brain data of the second patient. As another example, thedesired brain data 252 can be generated by combining i) current braindata of the patient (i.e., before the treatment is provided to thepatient) and ii) brain data of the second patient.

In some other implementations, the system can determine the desiredbrain data 252 of the patient using only the current brain data of thepatient, i.e., by modifying some or all of the elements of the currentbrain data of the patient to reflect the desired outcome.

The inverted autoencoder neural network system 250 can process thedesired brain data 252 to generate reconstructed desired brain data 272.In particular, the inverted autoencoder neural network system 250 can i)process the desired brain data 252 using the inverted decoder subnetwork260 to generate an embedding 262 of the desire brain data 252, and ii)process the embedding 262 of the desire brain data 252 using theinverted encoder subnetwork 270 to generate reconstructed desired braindata 272.

The reconstructed desired brain data 272 can characterize a predictedlocal effect of the future treatment on the brain of the patient. Thatis, the reconstructed desired brain data 272 can reflect the predictedeffect of the treatment, e.g., the short-term effect of the treatment,on the location at which the treatment will be targeted.

The reconstructed desired brain data 272 can be used to determineparameters of the future treatment such that the future treatment willactualize the desired global effect reflected in the desired brain data252. For example, the reconstructed desired brain data 272 can be usedto determine a recommended location in the brain of the patient totarget with the future treatment. As another example, the reconstructeddesired brain data 272 can be used to determine a recommended strengthor dose of the future treatment. As another example, the reconstructeddesired brain data 272 can be used to determine a recommended schedulefor providing the future treatment to the patient.

For example, an external system can identify one or more differencesbetween i) the reconstructed desired brain data 272 and ii) the currentbrain data of the patient. As a particular example, the external systemcan identify one or more elements in the reconstructed desired braindata 272 that have the largest difference between the correspondingelements of the current brain data of the patient. The external systemcan then recommend the target location of the future treatment to be alocation in the brain of the patient corresponding to the one or moredetermined differences. Instead or in addition, the external system canrecommend a strength or dose of the future treatment according to amagnitude of the one or more determined differences.

Thus, the reconstructed desired brain data 172 can also be called“roadmap” brain data because the reconstructed desired brain data 172can be used as a roadmap to determine a future treatment that, ifprovided to the patient, will, or will likely, actualize the desiredglobal effect.

As a particular example, the desired brain data 252 can include desiredcorrelation data 280 (e.g., as represented by a correlation matrix inFIG. 2B) that includes, for each of one or more pairs of parcellationsin the brain of the patient, a desired correlation between therespective brain activity in the pair of parcellations. The invertedautoencoder neural network system 250 can process the desiredcorrelation data 280 to generate roadmap correlation data 290 (e.g., asrepresented by a correlation matrix in FIG. 2B) that includes, for eachof the one or more pairs of parcellations in the brain of the patient, a“roadmap” correlation between the respective brain activity in the pairof parcellations. The external system can then identify a differencebetween the desired correlation data 280 and the current brain data ofthe patient in order to determine the parameters of a future treatment.For example, the external system can identify a particular parcellationwhose correlation values are different between the desired correlationdata 280 and the current brain data of the patient, and determine theparameters of a TMS treatment that will stimulate the particularparcellation.

The output of the inverted autoencoder neural network system 250 canrepresent the local effect of the future treatment because of thetraining of the autoencoder neural network 200, i.e., the network thatwas used to determine the inverted autoencoder neural network system250. As described above, the autoencoder neural network 200 can beconfigured through training to learn the relationships between differentportions of the input brain data 202 (e.g., relationships between thebrain data 202 corresponding to respective parcellations in the brain ofthe patient). Therefore, inverting the autoencoder neural network system200 can configure the inverted decoder subnetwork 260 to process inputbrain data 252 to generate embeddings 262 that encode the relationshipsbetween the portions of the brain data 252. The inverted encodersubnetwork 270 can then process the embeddings 262 to generatereconstructed brain data 272 that identifies changes in one or moreparticular portions of the brain data (i.e., local changes) that caninfluence the rest of the brain data such that the brain data reflectsthe desired characteristics of the input brain data 252. Furthermore,typically the differences between the brain data of different patientsare continuous and relatively standard, e.g., the values of a particularelement of brain data corresponding to respective patients can fallwithin a limited range. Therefore, the reconstructed brain data 272generated by the inverted autoencoder neural network 250 can identifystandard values for most elements (i.e., elements that would not haveinfluence on the desired characteristics of the input brain data 252),while identifying relatively abnormal values (corresponding to the localeffects of the future treatment) for the elements that would actualizethe particular characteristics of the input brain data 252.

In some implementations, the inverted autoencoder neural network system250 can generate multiple different sets of reconstructed desired braindata 272 from a single set of desired brain data 252. Because there issome loss of information when the autoencoder neural network system 200generates the embedding 212 for a particular set of brain data 202,multiple different sets of brain data 202 can correspond to a single setof reconstructed brain data 222. Thus, the inverted autoencoder neuralnetwork system 250 can determine multiple different sets ofreconstructed desired brain data 272 that, if processed by theautoencoder neural network system 200, would generate the same set ofdesired brain data 252.

After the inverted autoencoder neural network system 250 has generatedthe multiple different sets of reconstructed desired brain data 272, anexternal system can select a particular set from the multiple differentsets of reconstructed desired brain data 272 from which to determine theparameters of the future treatment. For example, the external system candetermine, for each of the multiple different sets, a difference betweeni) the set of reconstructed desired brain data 272 and ii) the currentbrain data of the patient. The external system can then select theparticular set from the multiple different sets according to therespective determined differences. As a particular example, the externalsystem can select the particular set that has a smallest difference fromthe current brain data of the patient, i.e., the particular set ofreconstructed desired brain data 272 that reflects the smallest localeffect that would induce the desired global effect and thus that maycorrespond to the least intensive treatment. As another particularexample, the external system can select the particular set that has themost localized differences from the current brain data of the patient,i.e., a particular set of reconstructed desired brain data 272 fromwhich a treatment can be determined that can target a single location inthe brain of the patient instead of multiple different locations.

In some other implementations, a system can generate reconstructeddesired brain data 272 by processing desired brain data 252 using agenerative neural network that has been trained adversarially. Forexample, the generative neural network can process the desired braindata 252 using one or more feedforward neural network layers to generatethe reconstructed brain data 272.

A training system can train the generative neural network using agenerative adversarial network that includes the generative neuralnetwork and one or more discriminator neural networks. The discriminatorneural networks are each configured to process the output of thegenerative neural network (in this example, a set of reconstructeddesired brain data 272) and to generate a prediction of whether thereconstructed desired brain data 272 is real or synthetic, i.e., whetherthe reconstructed desired brain data 272 has is (or has been generatedfrom) real brain data of a patient (e.g., real fMRI data that has beencaptured from the brain of the patient) or whether the reconstructeddesired brain data 272 has been generated by the generative neuralnetwork. Generally, the training system determines the updates to theparameters of the generative neural network in order to increase anerror in the respective predictions of the discriminator neuralnetworks, and determines updates to the parameters of the discriminatorneural networks in order to decrease their errors. Example generativeadversarial networks are discussed in more detail in the U.S. PatentApplication entitled “Predicting Brain Data using Machine LearningModels,” to Michael Sughrue, Stephane Doyen, and Peter Nicholas, filedon the same day as the present application and incorporated herein byreference in its entirety.

FIG. 3 is a diagram of an example anomaly detection system 300. Theanomaly detection system 300 is an example of systems implemented ascomputer programs on one or more computers in one or more locations, inwhich the systems, components, and techniques described below can beimplemented.

The anomaly detection system 300 is configured to detect anomalies inbrain data 302 characterizing the brain of a patient. As describedabove, the brain data 302 can include any data characterizing the brainof the patient. For example, the brain data 302 can include correlationdata that characterizes, for each pair of parcellations in the brain ofthe patient, a degree of correlation between the respective brainactivity of the parcellations. As another example, the brain data 302can include tractography data that characterizes neural tractsconnecting pairs of parcellations in the brain of the patient.

The anomaly detection system includes an autoencoder neural networksystem 310 and an anomaly detection system 320. The autoencoder neuralnetwork system is configured to process the brain data 302 and togenerate reconstructed brain data 312. For example, the autoencoderneural network system 310 can be the autoencoder neural network system200 described above with reference to FIG. 2A.

The anomaly detection engine 320 is configured to process thereconstructed brain data 312, and optionally the patient brain data 302,to determine whether the brain data 302 or 312 is anomalous, i.e.,outside of a range of normal values or otherwise dissimilar with typicalbrain data of patients.

In some implementations, the patient brain data 302 is real brain datacaptured from the brain of the patient (or generated from real braindata captured from the brain of the patient), and the anomaly detectionengine 320 processes i) the patient brain data 302 and ii) thereconstructed brain data 312 to determine whether the patient brain data302 is anomalous. In particular, the anomaly detection engine 320 candetermine whether the patient brain data 302 is anomalous according to adifference between the patient brain data 302 and the reconstructedbrain data 312.

For example, the anomaly detection engine 320 can determine thedifference to be the L₁ or L₂ distance between the brain data 302 andthe reconstructed brain data 312, or squared versions thereof.

In some implementations, the anomaly detection engine 320 can determinethat the patient brain data 302 is anomalous if the determineddifference between the patient brain data 302 and the reconstructedbrain data 312 exceeds a predetermined threshold. The predeterminedthreshold can be determined during or after training of the autoencoderneural network system 310. For example, after a training system hastrained the autoencoder neural network system 310, the training systemcan process multiple testing examples corresponding to respectivedifferent patients using the autoencoder neural network system 310, anddetermine, for each training example, the difference between the inputbrain data and the reconstructed brain data. The training system canthen determine the predetermined threshold such that the computeddifference for some percentage of the testing examples, e.g., 0.8, 0.9,0.95, or 0.99, are below the threshold while the computed difference forthe remaining testing examples are above the threshold.

The autoencoder neural network system 310 has been trained to generatereconstructed brain data 312 that is similar to the patient brain data302, using training example that include “normal” brain data, i.e.,brain data that is not anomalous, corresponding to respective differentpatients. Therefore, if the anomaly detection engine 320 determines thatthe reconstructed brain data 312 is dissimilar to the patient brain data302, according to the difference between the two, then the anomalydetection engine 320 can determine that the patient brain data 302 isanomalous.

In some other implementations, the patient brain data 302 is modifiedbrain data that characterizes a predicted local effect of a futuretreatment on the brain of the patient, as described above with referenceto FIG. 2A. The anomaly detection engine can process the correspondingreconstructed brain data 312, which characterizes a predicted globaleffect of the future treatment on the brain of the patient, to determinewhether the reconstructed brain data 312 is anomalous. That is, theanomaly detection engine 320 can determine whether providing the futuretreatment to the patient might cause one or more anomalies in the brainof the patient, which might indicate that the treatment may be dangerousfor the patient. In other words, when a clinician is considering atreatment for a patient, the clinician or another user can modify thepatient's brain data to include the predicted change as a result of theproposed treatment, producing patient-specific modified brain data. Thesystem can then i) apply the autoencoder to the patient-specificmodified brain data to produce reconstructed brain data, and ii) detectpotential anomalies in the reconstructed brain data (e.g., using any ofa variety of anomaly detection techniques).

In some such implementations, the anomaly detection engine 320 canprocess the reconstructed brain data 312 using a machine learning modelthat is configured to process reconstructed brain data 312 to determinewhether it is anomalous. For example, the machine learning modelgenerates a prediction characterizing a likelihood that thereconstructed brain data 312 is anomalous, e.g., a scalar value between0 and 1. As another example, the machine learning model identifies oneor more elements in the reconstructed brain data 312 that are anomalous.The machine learning model can be trained using brain data (e.g., realbrain data and/or reconstructed brain data) corresponding to multipledifferent patients.

In some other such implementations, the anomaly detection engine 320 candetermine whether the reconstructed brain data 312 is anomalous if thereconstructed brain data 312 is outside of a “normal” range of values asdefined by a set of brain data corresponding to other patients. Forexample, the anomaly detection engine 320 can obtain “normal” brain datathat identifies, for each of one or more elements in the reconstructedbrain data 312, a range of values for the element that is considerednormal. The normal brain data can be determined from brain data capturedfrom hundreds, thousands, or millions of other patients. As a particularexample, the normal brain data can identify, for each element in thereconstructed brain data 312, an average value and a standard deviationof values, as determined from the brain data of the other patients.

The anomaly detection engine 320 can use the normal brain data toidentify one or more elements of the reconstructed brain data 312 thatare anomalous. As a particular example, the anomaly detection engine 320might determine that the value of a particular element is anomalous ifthe value is outside a range defined by the average value and standarddeviation of values. For example, the value of an element can bedetermine to be anomalous if it is outside one, two, three, or fourstandard deviations of the average value.

The anomaly detection engine 320 can then determine whether the one ormore determined anomalous elements in the reconstructed brain data 312indicate that the future treatment may be unsafe. For example, theanomaly detection system can obtain data identifying one or moreparticular elements in the reconstructed brain data 312 that, if theyare determined to be anomalous, indicate an issue in the brain of thepatient and therefore that the future treatment may be unsafe.

In some implementations, the anomaly detection system 300 can providedata representing the one or more identified anomalous elements in thereconstructed brain data 312 to a downstream system for furtherprocessing. For example, the anomaly detection system 300 can providedata to a graphical user interface for displaying data representing theone or more anomalous elements to a user. As another example, theanomaly detection system 300 can provide data to a machine learningsystem that is configured to process a model input that includes thedata and generate a model output that is clinically relevant for a user.For example, a machine learning model can process the data to generate aprediction for whether the patient has a particular brain disease, e.g.,autism, depression, or schizophrenia.

Anomaly detection systems are described in more detail in U.S. patentapplication Ser. No. 16/920,078, filed on Jul. 2, 2020, the contents ofwhich are hereby incorporated by reference in their entirety.

FIG. 4 is a flowchart of an example process 400 for processing braindata using an autoencoder neural network. The process 400 can beimplemented by one or more computer programs installed on one or morecomputers and programmed in accordance with this specification. Forexample, the process 400 can be performed by the computer server moduledepicted in FIG. 1A. For convenience, the process 400 will be describedas being performed by a system of one or more computers.

The system obtains brain data captured from one or more sensorscharacterizing brain activity of a patient (step 402).

The system processes the brain data to generate modified brain data thatcharacterizes a predicted local effect of a future treatment on thebrain of the patient (step 404). For example, the system can determinethe target location of the future treatment in the brain of the patient,identify one or more elements of the brain data corresponding to thetarget location, and modify values of the one or more elements accordingto one or more parameters of the future treatment.

The system processes the modified brain data using an autoencoder neuralnetwork to generate reconstructed brain data (step 406). The system canprocess a network input that includes the modified brain data using anencoder subnetwork of the autoencoder neural network to generate anembedding of the network input, and then process the embedding of thenetwork input using a decoder subnetwork of the autoencoder neuralnetwork to generate the reconstructed brain data.

The system determines, using the reconstructed brain data, a predictedfuture global effect of the future treatment on the brain of the patient(step 408). For example, the system can process the reconstructed braindata to identify one or more anomalies in the reconstructed brain data.The identified anomalies might indicate that the future treatment isunsafe. As a particular example, the system can obtain normal brain datathat identifies, for each of one or more elements in the reconstructedbrain data, a respective normal range of values for the element. Thesystem can then compare i) the reconstructed brain data and ii) thenormal brain data to identify the one or more anomalies.

In some implementations, the system performs the process 400 multipletimes for respective different future treatments. The system can thenselect a particular future treatment form the multiple different futuretreatments using the respective predicted global effects of the multiplefuture treatments. For example, the system can select the particularfuture treatment that has a global effect that reflects the safest andmost effective outcome of the multiple different future treatments.

FIG. 5 is a flowchart of an example process 500 for processing braindata using an inverted autoencoder neural network. The process 500 canbe implemented by one or more computer programs installed on one or morecomputers and programmed in accordance with this specification. Forexample, the process 500 can be performed by the computer server moduledepicted in FIG. 1A. For convenience, the process 500 will be describedas being performed by a system of one or more computers.

The system obtains desired brain data characterizing desired brainactivity of a patient (step 502). The desired brain data cancharacterize a possible global effect of a future treatment on the brainof the patient. For example, the desired brain data can be the same as(or generated from) brain data of a second patient that characterizesthe desired brain activity.

The system provides the desired brain data as input to an invertedautoencoder neural network (step 504). The inverted autoencoder neuralnetwork can be determined using a trained autoencoder neural network,including inverting one or more operations of the autoencoder neuralnetwork.

The autoencoder can be configured to process brain data of a patient andto generate reconstructed brain data of the patient by i) processing anetwork input that includes the brain data using an encoder subnetworkto generate an embedding of the network input, and ii) processing theembedding of the network input using a decoder subnetwork to generatethe reconstructed brain data.

The system obtains, from the inverted neural network, roadmap brain datathat characterizes a predicted local effect of the future treatment onthe brain of the patient (step 506).

Optionally, the system can determine, from the roadmap data, respectivevalues for one or more parameters of the future treatment (step 508).The values for the one or more parameters can be determined such thatthe future treatment actualizes the desired brain activity of thepatient.

In some implementations, the system can obtain, from the inverted neuralnetwork, multiple different sets of roadmap data that each characterizea predicted local effect of the future treatment if the future treatmentwere to be provided to the patient according to respective differentvalues for one or more parameters of the future treatment. The systemcan then select particular values for the one or more parameters of thefuture treatment using the respective sets of roadmap data.

FIG. 6 is a flowchart of an example process 600 for identifyinganomalous brain data using an autoencoder neural network. The process600 can be implemented by one or more computer programs installed on oneor more computers and programmed in accordance with this specification.For example, the process 600 can be performed by the computer servermodule depicted in FIG. 1A. For convenience, the process 600 will bedescribed as being performed by a system of one or more computers.

The system obtains brain data captured from one or more sensorscharacterizing brain activity of a patient (step 602).

The system processes the brain data using an autoencoder neural networkto generate reconstructed brain data (step 604). The system can processa network input that includes the brain data using an encoder subnetworkof the autoencoder neural network to generate an embedding of thenetwork input, and then process the embedding of the network input usinga decoder subnetwork of the autoencoder neural network to generate thereconstructed brain data.

The system determines, using the reconstructed brain data, whether thebrain data of the patient is anomalous (step 606). For example, thesystem can determine an error between the brain data and thereconstructed brain data, and determine whether the error satisfies apredetermine threshold. The predetermined threshold can be determinedaccording to training errors, where each training error represents anerror between i) second brain data of a second patient, and ii) secondreconstructed brain data generated by processing the second brain datausing the autoencoder neural network.

Embodiments of the subject matter and the functional operationsdescribed in this specification can be implemented in digital electroniccircuitry, in tangibly-embodied computer software or firmware, incomputer hardware, including the structures disclosed in thisspecification and their structural equivalents, or in combinations ofone or more of them. Embodiments of the subject matter described in thisspecification can be implemented as one or more computer programs, i.e.,one or more modules of computer program instructions encoded on atangible non-transitory storage medium for execution by, or to controlthe operation of, data processing apparatus. The computer storage mediumcan be a machine-readable storage device, a machine-readable storagesubstrate, a random or serial access memory device, or a combination ofone or more of them. Alternatively or in addition, the programinstructions can be encoded on an artificially-generated propagatedsignal, e.g., a machine-generated electrical, optical, orelectromagnetic signal, that is generated to encode information fortransmission to suitable receiver apparatus for execution by a dataprocessing apparatus.

The term “data processing apparatus” refers to data processing hardwareand encompasses all kinds of apparatus, devices, and machines forprocessing data, including by way of example a programmable processor, acomputer, or multiple processors or computers. The apparatus can alsobe, or further include, special purpose logic circuitry, e.g., an FPGA(field programmable gate array) or an ASIC (application-specificintegrated circuit). The apparatus can optionally include, in additionto hardware, code that creates an execution environment for computerprograms, e.g., code that constitutes processor firmware, a protocolstack, a database management system, an operating system, or acombination of one or more of them.

A computer program which may also be referred to or described as aprogram, software, a software application, an app, a module, a softwaremodule, a script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, or declarative orprocedural languages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, or other unitsuitable for use in a computing environment. A program may, but neednot, correspond to a file in a file system. A program can be stored in aportion of a file that holds other programs or data, e.g., one or morescripts stored in a markup language document, in a single file dedicatedto the program in question, or in multiple coordinated files, e.g.,files that store one or more modules, sub-programs, or portions of code.A computer program can be deployed to be executed on one computer or onmultiple computers that are located at one site or distributed acrossmultiple sites and interconnected by a data communication network.

For a system of one or more computers to be configured to performparticular operations or actions means that the system has installed onit software, firmware, hardware, or a combination of them that inoperation cause the system to perform the operations or actions. For oneor more computer programs to be configured to perform particularoperations or actions means that the one or more programs includeinstructions that, when executed by data processing apparatus, cause theapparatus to perform the operations or actions.

As used in this specification, an “engine,” or “software engine,” refersto a software implemented input/output system that provides an outputthat is different from the input. An engine can be an encoded block offunctionality, such as a library, a platform, a software development kit(“SDK”), or an object. Each engine can be implemented on any appropriatetype of computing device, e.g., servers, mobile phones, tabletcomputers, notebook computers, music players, e-book readers, laptop ordesktop computers, PDAs, smart phones, or other stationary or portabledevices, that includes one or more processors and computer readablemedia. Additionally, two or more of the engines may be implemented onthe same computing device, or on different computing devices.

The processes and logic flows described in this specification can beperformed by one or more programmable computers executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby special purpose logic circuitry, e.g., an FPGA or an ASIC, or by acombination of special purpose logic circuitry and one or moreprogrammed computers.

Computers suitable for the execution of a computer program can be basedon general or special purpose microprocessors or both, or any other kindof central processing unit. Generally, a central processing unit willreceive instructions and data from a read-only memory or a random accessmemory or both. The essential elements of a computer are a centralprocessing unit for performing or executing instructions and one or morememory devices for storing instructions and data. The central processingunit and the memory can be supplemented by, or incorporated in, specialpurpose logic circuitry. Generally, a computer will also include, or beoperatively coupled to receive data from or transfer data to, or both,one or more mass storage devices for storing data, e.g., magnetic,magneto-optical disks, or optical disks. However, a computer need nothave such devices. Moreover, a computer can be embedded in anotherdevice, e.g., a mobile telephone, a personal digital assistant (PDA), amobile audio or video player, a game console, a Global PositioningSystem (GPS) receiver, or a portable storage device, e.g., a universalserial bus (USB) flash drive, to name just a few.

Computer-readable media suitable for storing computer programinstructions and data include all forms of non-volatile memory, mediaand memory devices, including by way of example semiconductor memorydevices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks,e.g., internal hard disks or removable disks; magneto-optical disks; andCD-ROM and DVD-ROM disks.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on a computerhaving a display device, e.g., a CRT (cathode ray tube) or LCD (liquidcrystal display) monitor, for displaying information to the user and akeyboard and pointing device, e.g, a mouse, trackball, or a presencesensitive display or other surface by which the user can provide inputto the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback, e.g., visual feedback,auditory feedback, or tactile feedback; and input from the user can bereceived in any form, including acoustic, speech, or tactile input. Inaddition, a computer can interact with a user by sending documents toand receiving documents from a device that is used by the user; forexample, by sending web pages to a web browser on a user's device inresponse to requests received from the web browser. Also, a computer caninteract with a user by sending text messages or other forms of messageto a personal device, e.g., a smartphone, running a messagingapplication, and receiving responsive messages from the user in return.

Embodiments of the subject matter described in this specification can beimplemented in a computing system that includes a back-end component,e.g., as a data server, or that includes a middleware component, e.g.,an application server, or that includes a front-end component, e.g., aclient computer having a graphical user interface, a web browser, or anapp through which a user can interact with an implementation of thesubject matter described in this specification, or any combination ofone or more such back-end, middleware, or front-end components. Thecomponents of the system can be interconnected by any form or medium ofdigital data communication, e.g., a communication network. Examples ofcommunication networks include a local area network (LAN) and a widearea network (WAN), e.g., the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someembodiments, a server transmits data, e.g., an HTML page, to a userdevice, e.g., for purposes of displaying data to and receiving userinput from a user interacting with the device, which acts as a client.Data generated at the user device, e.g., a result of the userinteraction, can be received at the server from the device.

In addition to the embodiments described above, the followingembodiments are also innovative:

Embodiment 1 is a method comprising:

obtaining brain data captured by one or more sensors characterizingbrain activity of a patient;

processing the brain data to generate modified brain data thatcharacterizes a predicted local effect of a future treatment on thebrain of the patient, wherein the local effect of the future treatmentis an effect of the future treatment on a region of the brain that islocal to a target location of the future treatment in the brain;

processing the modified brain data using an autoencoder neural networkto generate reconstructed brain data, wherein:

-   -   the processing comprises:        -   processing a network input comprising the modified brain            data using an encoder subnetwork to generate an embedding of            the network input, and        -   processing the embedding of the network input using a            decoder subnetwork to generate the reconstructed brain data;            and    -   the autoencoder neural network has been trained by, at each of a        plurality of training time steps:        -   processing, according to current values of a plurality of            network parameters of the autoencoder neural network, a            training network input comprising second brain data of a            second patient to generate reconstructed second brain data,            and        -   determining an update to the current values of the plurality            of network parameters according to an error between i) the            second brain data and ii) the reconstructed second brain            data; and

determining, using the reconstructed brain data, a predicted globaleffect of the future treatment on the brain of the patient, wherein theglobal effect of the future treatment is an effect of the futuretreatment on one or more regions of the brain that are not local to thetarget of the future treatment in the brain.

Embodiment 2 is the method of embodiment 1, wherein processing the braindata to generate modified brain data that characterizes a predictedlocal effect of a treatment on the brain of the patient comprises:

determining the target location of the future treatment in the brain ofthe patient;

identifying one or more elements of the brain data corresponding to thetarget location; and

modifying values of the one or more elements according to one or moreparameters of the future treatment.

Embodiment 3 is the method of any one of embodiments 1 or 2, whereindetermining the predicted global effect of the future treatmentcomprises processing the reconstructed brain data to identify one ormore anomalies in the reconstructed brain data.

Embodiment 4 is the method of embodiment 3, wherein identifying one ormore anomalies in the reconstructed brain data comprises:

obtaining normal brain data identifying, for each of one or moreelements in the reconstructed brain data, a respective normal range ofvalues for the element; and

comparing i) the reconstructed brain data and ii) the normal brain datato identify the one or more anomalies.

Embodiment 5 is the method of any one of embodiments 1-4, furthercomprising:

for each of one or more second future treatments:

-   -   processing the brain data to generate second modified brain data        that characterizes a predicted local effect of the second future        treatment on the brain of the patient;    -   processing the second modified brain data using the autoencoder        neural network to generate second reconstructed brain data; and    -   determining, using the second reconstructed brain data, a        predicted global effect of the second future treatment on the        brain of the patient; and

selecting, using the respective predicted global effects of the futuretreatment and the one or more second future treatments, a particularfuture treatment.

Embodiment 6 is the method of any one of embodiments 1-5, wherein thefuture treatment is a transcranial magnetic stimulation (TMS) treatment.

Embodiment 7 is the method of any one of embodiments 1-6, wherein:

the brain data comprises correlation data characterizing, for each of aplurality of pairs of parcellations formed from a set of parcellationsin the brain of the patient, where each pair comprises a firstparcellation and a second parcellation, a degree of correlation betweenthe brain activity of the first parcellation and the brain activity ofthe second parcellation in the brain of the patient.

Embodiment 8 is a method comprising:

obtaining desired brain data characterizing desired brain activity of apatient, wherein the desired brain data characterizes a possible globaleffect of a future treatment on the brain of the patient, wherein theglobal effect of the treatment is an effect of the treatment on one ormore regions of the brain that are not local to a target of thetreatment in the brain;

providing the desired brain data as input to an inverted autoencoderneural network, wherein:

-   -   the inverted autoencoder neural network has been determined        using a trained autoencoder neural network, the determining        comprising inverting a plurality of operations of the        autoencoder neural network;    -   the autoencoder neural network is configured to process brain        data of a patient and to generate reconstructed brain data of        the patient, the processing comprising:        -   processing a network input comprising the brain data using            an encoder subnetwork to generate an embedding of the            network input, and        -   processing the embedding of the network input using a            decoder subnetwork to generate the reconstructed brain data;            and    -   the autoencoder neural network has been trained by, at each of a        plurality of training time steps:        -   processing, according to current values of a plurality of            network parameters of the autoencoder neural network, a            training network input comprising second brain data of a            second patient to generate reconstructed second brain data,            and        -   determining an update to the current values of the plurality            of network parameters according to an error between i) the            second brain data and ii) the reconstructed second brain            data; and

obtaining, from the inverted autoencoder neural network, roadmap braindata that characterizes a predicted local effect of the future treatmenton the brain of the patient, wherein the local effect of the futuretreatment is an effect of the future treatment on a region of the brainthat is local to the target of the treatment.

Embodiment 9 is the method of embodiment 8, further comprising:

determining, from the roadmap brain data, respective values for one ormore parameters of the future treatment in order to actualize thedesired brain activity of the patient.

Embodiment 10 is the method of embodiment 9, wherein the parameters ofthe future treatment comprise one or more of:

a recommended target location of the feature treatment,

a recommended strength of the future treatment,

a recommended dose of the future treatment; or

a recommended schedule of the future treatment.

Embodiment 11 is the method of any one of embodiments 8-10, whereinobtaining desired brain data characterizing desired brain activity of apatient comprises obtaining brain data of a second patient thatcharacterizes the desired brain activity.

Embodiment 12 is the method of any one of embodiments 8-11, wherein:

the roadmap data characterizes a predicted local effect of the futuretreatment on the brain of the patient if the future treatment wereprovided according to first values for one or more parameters of thefuture treatment; and

the method further comprises:

-   -   obtaining, from the inverted autoencoder neural network, one or        more sets of second roadmap brain data that each characterize a        predicted local effect of the future treatment on the brain of        the patient if the future treatment were provided according to        respective second values of the one or more parameters of the        future treatment; and    -   selecting particular values for the one or more parameters of        the future treatment using the roadmap brain data and the one or        more sets of second roadmap data.

Embodiment 13 is the method of any one of embodiments 8-12, wherein thefuture treatment is a transcranial magnetic stimulation (TMS) treatment.

Embodiment 14 is the method of any one of embodiments 8-13, wherein:

the brain data comprises correlation data characterizing, for each of aplurality of pairs of parcellations formed from a set of parcellationsin the brain of the patient, where each pair comprises a firstparcellation and a second parcellation, a degree of correlation betweenthe brain activity of the first parcellation and the brain activity ofthe second parcellation in the brain of the patient.

Embodiment 15 is a method comprising:

obtaining brain data generated from data captured by one or more sensorscharacterizing brain activity of a patient;

processing the brain data using an autoencoder neural network togenerate reconstructed brain data, wherein:

-   -   the processing comprises:        -   processing a network input comprising the brain data using            an encoder subnetwork to generate an embedding of the            network input, and        -   processing the embedding of the network input using a            decoder subnetwork to generate the reconstructed brain data;            and    -   the autoencoder neural network has been trained by, at each of a        plurality of training time steps:        -   processing, according to current values of a plurality of            network parameters of the autoencoder neural network, a            training network input comprising second brain data of a            second patient to generate reconstructed second brain data,            and        -   determining an update to the current values of the plurality            of network parameters according to an error between i) the            second brain data and ii) the reconstructed second brain            data; and

determining, using the reconstructed brain data, whether the brain dataof the patient is anomalous.

Embodiment 16 is the method of embodiment 15, wherein determiningwhether the brain data of the patient is anomalous comprises:

determining an error between the brain data and the reconstructed braindata; and

determining whether the error satisfies a predetermined threshold.

Embodiment 17 is the method of embodiment 16, wherein the threshold isdetermined according to a plurality of training errors, wherein eachtraining error represents an error between i) second brain data of asecond patient, and ii) second reconstructed brain data generated byprocessing the second brain data using the autoencoder neural network.

Embodiment 18 is the method of any one of embodiments 15-17, wherein:

the brain data comprises correlation data characterizing, for each of aplurality of pairs of parcellations formed from a set of parcellationsin the brain of the patient, where each pair comprises a firstparcellation and a second parcellation, a degree of correlation betweenthe brain activity of the first parcellation and the brain activity ofthe second parcellation in the brain of the patient.

Embodiment 19 is the method of any one of embodiments 15-18, wherein thebrain data is modified brain data that characterizes a predicted localeffect of a future treatment on the brain of the patient, wherein thelocal effect of the future treatment is an effect of the futuretreatment on a region of the brain that is local to a target location ofthe future treatment in the brain.

Embodiment 20 is the method of embodiment 19, wherein determiningwhether the brain data of the patient is anomalous comprises:

obtaining normal brain data identifying, for each of one or moreparticular elements of the brain data of the patient, a respectivenormal range of values for the particular element; and

comparing i) the reconstructed brain data and ii) the normal brain datato identify the one or more possible anomalies.

Embodiment 21 is the method of embodiment 20, wherein:

the normal brain data is generated from a plurality of sets of braindata corresponding to respective other patients;

the normal brain data comprises, for each of the one or more particularelements of the brain data of the patient, data characterizing i) ameasure of central tendency of the value of the particular element andii) a measure of variance of the value of the particular element,wherein the measure of central tendency and the measure of variance havebeen computed using the plurality of sets of brain data; and

for each of the one or more particular elements of the brain data of thepatient, the normal range of values for the particular element isdefined by a maximum value and a minimum value, wherein the maximumvalue and the minimum value are linear combinations of the correspondingmeasure of central tendency of the particular element and thecorresponding measure of variance of the particular element.

Embodiment 22 is a system comprising one or more computers and one ormore storage devices storing instructions that are operable, whenexecuted by the one or more computers, to cause the one or morecomputers to perform the method of any one of embodiments 1-21.

Embodiment 23 is one or more non-transitory computer storage mediumencoded with a computer program, the program comprising instructionsthat are operable, when executed by data processing apparatus, to causethe data processing apparatus to perform the method of any one ofembodiments 1-21.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinvention or on the scope of what may be claimed, but rather asdescriptions of features that may be specific to particular embodimentsof particular inventions. Certain features that are described in thisspecification in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable subcombination. Moreover, although features may be describedabove as acting in certain combinations and even initially be claimed assuch, one or more features from a claimed combination can in some casesbe excised from the combination, and the claimed combination may bedirected to a subcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various system modulesand components in the embodiments described above should not beunderstood as requiring such separation in all embodiments, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims. For example,the actions recited in the claims can be performed in a different orderand still achieve desirable results. As one example, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In certain some cases, multitasking and parallel processing maybe advantageous.

What is claimed is:
 1. A method comprising: obtaining brain datacaptured by one or more sensors characterizing brain activity of apatient; processing the brain data to generate modified brain data thatcharacterizes a predicted local effect of a future treatment on thebrain of the patient, wherein the predicted local effect is a predictionof an effect of the future treatment on a region of the brain that islocal to a target location of the future treatment in the brain;processing the modified brain data using an autoencoder neural networkto generate reconstructed brain data, wherein: the processing comprises:processing a network input generated from the modified brain data usingan encoder subnetwork to generate an embedding of the network input, andprocessing the embedding of the network input using a decoder subnetworkto generate the reconstructed brain data; and the autoencoder neuralnetwork has been configured through training to process the networkinput and to generate reconstructed brain data that characterizes apredicted global effect of the future treatment on the brain of thepatient, wherein the predicted global effect is a prediction of aneffect of the future treatment on at least one or more regions of thebrain that are not local to the target location of the future treatmentin the brain, the training comprising, at each of a plurality oftraining time steps: obtaining second brain data captured by one or moresecond sensors characterizing brain activity of a second patient;generating, from the obtained second brain data, a training networkinput; processing, according to current values of a plurality of networkparameters of the autoencoder neural network, the training network inputto generate reconstructed second brain data, and determining an updateto the current values of the plurality of network parameters accordingto an error between i) the second brain data and ii) the reconstructedsecond brain data; and determining, using the reconstructed brain data,the predicted global effect of the future treatment on the brain of thepatient.
 2. The method of claim 1, wherein processing the brain data togenerate modified brain data that characterizes a predicted local effectof a treatment on the brain of the patient comprises: determining thetarget location of the future treatment in the brain of the patient;identifying one or more elements of the brain data corresponding to thetarget location; and modifying values of the one or more elementsaccording to one or more parameters of the future treatment.
 3. Themethod of claim 1, wherein determining the predicted global effect ofthe future treatment comprises processing the reconstructed brain datato identify one or more anomalies in the reconstructed brain data. 4.The method of claim 3, wherein identifying one or more anomalies in thereconstructed brain data comprises: obtaining normal brain dataidentifying, for each of one or more elements in the reconstructed braindata, a respective normal range of values for the element; and comparingi) the reconstructed brain data and ii) the normal brain data toidentify the one or more anomalies.
 5. The method of claim 1, furthercomprising: for each of one or more second future treatments: processingthe brain data to generate second modified brain data that characterizesa predicted local effect of the second future treatment on the brain ofthe patient; processing the second modified brain data using theautoencoder neural network to generate second reconstructed brain data;and determining, using the second reconstructed brain data, a predictedglobal effect of the second future treatment on the brain of thepatient; and selecting, using the respective predicted global effects ofthe future treatment and the one or more second future treatments, aparticular future treatment.
 6. The method of claim 1, wherein thefuture treatment is a transcranial magnetic stimulation (TMS) treatment.7. The method of claim 1, wherein: the brain data comprises correlationdata characterizing, for each of a plurality of pairs of parcellationsformed from a set of parcellations in the brain of the patient, whereeach pair comprises a first parcellation and a second parcellation, adegree of correlation between the brain activity of the firstparcellation and the brain activity of the second parcellation in thebrain of the patient.
 8. A system comprising one or more computers andone or more storage devices storing instructions that are operable, whenexecuted by the one or more computers, to cause the one or morecomputers to perform operations comprising: obtaining brain datacaptured by one or more sensors characterizing brain activity of apatient; processing the brain data to generate modified brain data thatcharacterizes a predicted local effect of a future treatment on thebrain of the patient, wherein the predicted local effect is a predictionof an effect of the future treatment on a region of the brain that islocal to a target location of the future treatment in the brain;processing the modified brain data using an autoencoder neural networkto generate reconstructed brain data, wherein: the processing comprises:processing a network input generated from the modified brain data usingan encoder subnetwork to generate an embedding of the network input, andprocessing the embedding of the network input using a decoder subnetworkto generate the reconstructed brain data; and the autoencoder neuralnetwork has been configured through training to process the networkinput and to generate reconstructed brain data that characterizes apredicted global effect of the future treatment on the brain of thepatient, wherein the predicted global effect is a prediction of aneffect of the future treatment on at least one or more regions of thebrain that are not local to the target location of the future treatmentin the brain, the training comprising, at each of a plurality oftraining time steps: obtaining second brain data captured by one or moresecond sensors characterizing brain activity of a second patient;generating, from the obtained second brain data, a training networkinput; processing, according to current values of a plurality of networkparameters of the autoencoder neural network, the training network inputto generate reconstructed second brain data, and determining an updateto the current values of the plurality of network parameters accordingto an error between i) the second brain data and ii) the reconstructedsecond brain data; and determining, using the reconstructed brain data,the predicted global effect of the future treatment on the brain of thepatient.
 9. The system of claim 8, wherein processing the brain data togenerate modified brain data that characterizes a predicted local effectof a treatment on the brain of the patient comprises: determining thetarget location of the future treatment in the brain of the patient;identifying one or more elements of the brain data corresponding to thetarget location; and modifying values of the one or more elementsaccording to one or more parameters of the future treatment.
 10. Thesystem of claim 8, wherein determining the predicted global effect ofthe future treatment comprises processing the reconstructed brain datato identify one or more anomalies in the reconstructed brain data. 11.The system of claim 10, wherein identifying one or more anomalies in thereconstructed brain data comprises: obtaining normal brain dataidentifying, for each of one or more elements in the reconstructed braindata, a respective normal range of values for the element; and comparingi) the reconstructed brain data and ii) the normal brain data toidentify the one or more anomalies.
 12. The system of claim 8, theoperations further comprising: for each of one or more second futuretreatments: processing the brain data to generate second modified braindata that characterizes a predicted local effect of the second futuretreatment on the brain of the patient; processing the second modifiedbrain data using the autoencoder neural network to generate secondreconstructed brain data; and determining, using the secondreconstructed brain data, a predicted global effect of the second futuretreatment on the brain of the patient; and selecting, using therespective predicted global effects of the future treatment and the oneor more second future treatments, a particular future treatment.
 13. Thesystem of claim 8, wherein the future treatment is a transcranialmagnetic stimulation (TMS) treatment.
 14. The system of claim 8,wherein: the brain data comprises correlation data characterizing, foreach of a plurality of pairs of parcellations formed from a set ofparcellations in the brain of the patient, where each pair comprises afirst parcellation and a second parcellation, a degree of correlationbetween the brain activity of the first parcellation and the brainactivity of the second parcellation in the brain of the patient.
 15. Oneor more non-transitory storage media storing instructions that whenexecuted by one or more computers cause the one or more computers toperform operations comprising: obtaining brain data captured by one ormore sensors characterizing brain activity of a patient; processing thebrain data to generate modified brain data that characterizes apredicted local effect of a future treatment on the brain of thepatient, wherein the predicted local effect is a prediction of an effectof the future treatment on a region of the brain that is local to atarget location of the future treatment in the brain; processing themodified brain data using an autoencoder neural network to generatereconstructed brain data, wherein: the processing comprises: processinga network input generated from the modified brain data using an encodersubnetwork to generate an embedding of the network input, and processingthe embedding of the network input using a decoder subnetwork togenerate the reconstructed brain data; and the autoencoder neuralnetwork has been configured through training to process the networkinput and to generate reconstructed brain data that characterizes apredicted global effect of the future treatment on the brain of thepatient, wherein the predicted global effect is a prediction of aneffect of the future treatment on at least one or more regions of thebrain that are not local to the target location of the future treatmentin the brain, the training comprising, at each of a plurality oftraining time steps: obtaining second brain data captured by one or moresecond sensors characterizing brain activity of a second patient;generating, from the obtained second brain data, a training networkinput; processing, according to current values of a plurality of networkparameters of the autoencoder neural network, the training network inputto generate reconstructed second brain data, and determining an updateto the current values of the plurality of network parameters accordingto an error between i) the second brain data and ii) the reconstructedsecond brain data; and determining, using the reconstructed brain data,the predicted global effect of the future treatment on the brain of thepatient.
 16. The non-transitory storage media of claim 15, whereinprocessing the brain data to generate modified brain data thatcharacterizes a predicted local effect of a treatment on the brain ofthe patient comprises: determining the target location of the futuretreatment in the brain of the patient; identifying one or more elementsof the brain data corresponding to the target location; and modifyingvalues of the one or more elements according to one or more parametersof the future treatment.
 17. The non-transitory storage media of claim15, wherein determining the predicted global effect of the futuretreatment comprises processing the reconstructed brain data to identifyone or more anomalies in the reconstructed brain data.
 18. Thenon-transitory storage media of claim 17, wherein identifying one ormore anomalies in the reconstructed brain data comprises: obtainingnormal brain data identifying, for each of one or more elements in thereconstructed brain data, a respective normal range of values for theelement; and comparing i) the reconstructed brain data and ii) thenormal brain data to identify the one or more anomalies.
 19. Thenon-transitory storage media of claim 15, the operations furthercomprising: for each of one or more second future treatments: processingthe brain data to generate second modified brain data that characterizesa predicted local effect of the second future treatment on the brain ofthe patient; processing the second modified brain data using theautoencoder neural network to generate second reconstructed brain data;and determining, using the second reconstructed brain data, a predictedglobal effect of the second future treatment on the brain of thepatient; and selecting, using the respective predicted global effects ofthe future treatment and the one or more second future treatments, aparticular future treatment.
 20. The non-transitory storage media ofclaim 15, wherein: the brain data comprises correlation datacharacterizing, for each of a plurality of pairs of parcellations formedfrom a set of parcellations in the brain of the patient, where each paircomprises a first parcellation and a second parcellation, a degree ofcorrelation between the brain activity of the first parcellation and thebrain activity of the second parcellation in the brain of the patient.